Symbolic Neural Symbolic Algorithm/ Algorithm Network Neural Network

Neural networks offer an intriguing set of techniques for learning based on the ad-justment of weights of connections between processing units. However, the powerand limitations of connectionist methods for learning, such as the method of backpropagation in parallel distributed processing networks, are not yet entirely clear.We report on a set of experiments that more precisely identify the power and limi-tations of the method of back propagation. The experiment on learning to computethe exclusive-OR function suggests that the computational efficiency of learning bythe method of back propagation depends on the initial weights in the network. Theexperiment on learning to play Tic-Tac-Toe suggests that the information content ofwhat is learned by the back propagation method is dependent on the initial abstrac-tions in the network. It also suggests that these abstractions are a major source ofpower for learning in parallel distributed processing networks. In addition, we showthat the learning task addressed by connectionist methods, including the back prop-agation method, is computationally intractable. These experimental and theoreticalresults strongly indicate that current connectionist methods may be too limited forthe complex task of learning they seek to solve. We propose that the power of neuralnetworks may be enhanced by developing task-specific connectionist methods.